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ABSTRACT 

This technical report summarizes the third independent factor analysis of the SERU/UCUES questionnaire responses 
of students with majors. The 2009 solution employed the same quantitative analysis used in the prior solutions — 
varimax orthogonal rotation to determine principal components followed by promax oblique rotation to identify 
subfactors — supplemented by the collaborative judgment of a research team of faculty, institutional researchers, and 
a graduate student. Overall the three solutions have been remarkably similar. The most significant and substantive 
difference in factor structure was between 2006 and 2008 and was the result of several academic experience items 
being moved from a 2006 module to the core segment of the questionnaire. The most substantive differences from 
2008 to 2009 were methodological: respondents from several AAU institutions were included, factor analysis was 
performed on a weighted random sample of University of California and participating AAU students, factor scores 
were computed using item loadings, and a reduced set of factor scores was computed for all students, whether they 
had declared majors or not. 

The 2009 structure is comprised of eight principal factors, each with two or more subfactors. The principal component 
factors are: Factor 1: Satisfaction with Educational Experience; Factor 2: Current Skills Self-Assessment 
(Nonquantitative); Factor 3: Engagement with Studies; Factor 4: Gains in Self-Assessment of Skills (Nonquantitative); 
Factor 5: Development of Scholarship; Factor 6: Campus Climate for Diversity; Factor 7: Academic Disengagement 
(Inverted Scale); Factor 8: Quantitative Professions. 



Introduction 

Random Sample 

Factor analysis was performed on a simple random sample without replacement of students with declared 
majors weighted by institution. For the institutions in this project those weights were: Florida (1,675), 
Michigan (1,194), Minnesota (1,093), Pittsburgh (618), Rutgers University (1,175), Berkeley (902), Davis 
(1,047), UCLA (1,062), Riverside (364), San Diego (1,095), Santa Cruz (482), Santa Barbara (694), and 
Irvine (1,100). The sample was about 15,000 in order to provide about 12,500 valid responses for each 
factor. The sample was weighted by the number of undergraduates at each institution assigned to 
evaluate their major(s), and about half were from the AAU group. The student responses for Berkeley, 
which participated in both the 2008 and 2009 administrations, were taken from the 2008 UCUES 
administration where Berkeley had a higher response rate. Overall, the combined UC/AAU sample used 
in the current study should be much more robust and can be generalized to a broader population than 
was true for the prior solutions. 
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Summary of Changes to Factor Structure from 2008 to 2009 

The subfactor structure of Self Assessment of Skills (Nonquantitative), both current proficiency and gain 
scores, was changed from two to three subfactors. Factor 2, current skills, and Factor 4, gain in skills, 
now include a Computer and Research Skills subfactor. The new Computer and Research Skills 
subfactor broke out cleanly in 2009. Though it is not clear why these two subfactors did not emerge from 
the 2008 data, the result of their presence in the 2009 study can increase the value of these data to 
departments undergoing academic program review of Factors 2 and 4. 

The promax rotation to determine subfactors for Campus Climate for Diversity, Factor 6, found that 
agreement with the statement that students were respected regardless of sexual orientation loaded on 
two subfactors in 2009 (Table 6). Whereas in 2008, it fell with ratings of climate for SES, gender, and race 
items on the subfactor for personal characteristics, in 2009 it also loaded with religious and political 
beliefs on the climate of respect for personal beliefs subfactor. One potential explanation is that one 
sample views sexual orientation as a matter of choice, like religious or political beliefs, while the other 
sample did not view sexual orientation in terms of choice. Given the politically sensitive nature of the item 
and current understanding of human sexuality, the item was placed with subfactor 6a: Climate for 
Personal Characteristics. 

Factor 7, Academic Disengagement (Inverted Scale), now includes amount of reading. The item asking 
students to report the percentage of assigned reading that they completed was not included on the 
solution structure for 2008. It was reverse coded in 2009 and clearly contributed to the principal 
component and to the subfactor Poor Academic Habits (Table 7). Factor 7 was also changed in 2009 to 
an inverted scale so that higher values on all factors are better outcomes. 

Factor 1: Satisfaction with Educational Experience, subfactor Id: Satisfaction with Advising and Out of 
Class Contact, which was only an advising item, now includes two items that were previously with Clarity 
of Program Requirements, Policies and Practices and one from Duality of Instruction and Courses in the 
Major (Table 1). The three items being moved are communication channels being open, students being 
treated fairly, and faculty providing useful and prompt feedback. The movement of these items makes the 
prior subfactor more clearly about policies and broadens the advising subfactor to something more 
generally about communication. 

Computing Individual Student Factor Scores 

The process of computing factor and subfactor scores in 2009 extended the methodology used in prior 
years to include factor loadings. Briefly, the steps used in computing the factor score include: (1) 
responses by item are standardized to adjust for item response option types, (2) the mean of items in the 
factor or subfactor are computed for those standardized responses, and (3) the resulting factor scores are 
standardized and reported on a scale with a mean of 5 and standard deviation of 2. 

While the positive effect of this change must be determined by empirical evidence, the use of the factor 
loading to weight item contributions to the score should result in more accurate and effective factor 
scores. Because the loadings are decimal values between 0 and 1 and standard scores have a mean of 0 
and standard deviation of 1, large weights have little effect on the value while small weights tend to move 
standard scores toward the mean. For example, if a student’s standard score on item 1 is 0.8 and the 
weight of item 1 is 0.9, then the contribution of item 1 to the factor score is little changed (0.8 x 0.9 = 
0.72). If the weight is 0.3, then the contribution of item 1 has changed markedly (0.8 x 0.3 = 0.24) and is 
moved toward the mean of zero. Similarly, negative values are also moved toward the mean (e.g. -0.8 x 
0.3 

-0.24). 



New in 2009, factor scores were computed for all students whether they had declared majors or not. It 
continues to be true that the primary use of SERU/UCUES factor scores is in academic program review, 
and therefore, the factor structure, loadings, and standardization were based on students with declared 
majors. Flowever, in 2009 the decision was made to compute factor and subfactor scores for all students 
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where computations were supported by a full complement of items. Students without declared majors 
answered somewhat different questions and therefore principal components. 1: Satisfaction with 
Educational Experience, 7: Academic Disengagement, and 8: Quantitative Professions could not be 
computed for students with undeclared majors. In Factor 1, two subfactors could not be computed: la: 
Quality of Instruction and Courses in the Major and 1e: Clarity of Program Requirements, Policies, and 
Practices. Subfactor 7c: Non-academic Motivations, and Subfactor 8a: Career Orientation, could not be 
computed. Researchers should use these scores with caution because there is not current evidence of 
their validity in applied research, nor were they part of the population sampled for the factor analysis and 
factor score computation processes. 



Reliability as Measured by Coefficient Alpha (Table A) 

Internal consistency of factors as measured by Cronbach’s coefficient alpha ranged from 0.92 for 
Satisfaction with Educational Experience (Factor 1) to 0.61 for Quantitative Professions (Factor 8) and all 
subfactor reliability estimates were higher than 0.50. As was true of the factor solution, all reliability 
estimates of factors and subfactors were remarkably consistent over time and in no instance did the 
difference from year to year exceed 0.05. 



The Factors 

Factor 1: Satisfaction with Educational Experience (Table 1) 

Factor 1 is an exception to the rule that subfactors were formed by promax solution with oblique rotations. 
In fact, all items in Satisfaction with Educational Experience load heavily on one vector. The subscales 
offered for Factor 1 were created by a panel of experts developing the first solution for the 2006 data 
(Chatman, 2007). The subfactor structure is supported by factor analysis but is equally driven by the 
desire to provide useful composite measures: instruction, availability, belonging, advising, etc. Again, the 
decisions were not psychometrically arbitrary and the items do tend to have high internal consistency 
(Table A). In many respects, they are akin to Pike's notion of scalets and the use in NSSE as 
“Benchmarks” (see Pike’s discussion of scalets, 2006). 

Factor 2 and Factor 4: Current Skills Self-Assessment (Nonquantitative) and Gains in Self-Assessment of 
Skills (Nonquantitative) 

The SERU/UCUES questionnaire uses many self-rating items that ask students to assess their skills now 
and at entry to their campus. These values are used to compute a gain score, and the strategy has been 
shown to be more valid than asking students to assess gain in a more straightforward manner (see 
discussion in Thomson and Douglass, 2009, pp. 5-7). As mentioned previously, the 2009 solution found a 
new subfactor, Computer and Research Skills, that greatly improved the usefulness of Factor 2 scores. 
Items asking about Internet, computer, library research, and other research skills now comprise a 
subfactor distinct from the more general skills of Critical Thinking and Communication (see Table 2). 
There were slight differences in subfactor composition between the current skills ratings (Factor 2) and 
gain scores (Factor 4; see Table 4) but they were judged by the 2007 panel (Chatman, 2007) to be of 
minor importance and certainly of lesser importance than the need to report parallel measures. 

Factor 3: Engagement with Studies 

Factor 3 is one of a few factor scores that are similar in meaning and composition to NSSE benchmarks. 
It consists of three subfactors addressing academic involvement, initiative, research engagement, and 
collaborative work (Table 3). Engagement with Studies is a factor that is of special interest to faculty as it 
helps to describe students that many faculty, especially in social sciences and humanities, find to be more 
challenging and engaging for them. The scores in Factor 3, like NSSE benchmarks, tend to favor 
humanities and social sciences students and to penalize hard science and engineering students. 



Factor 5: Development of Scholarship 
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The majority of items in Factor 5 (see Table 5) were constructed to reflect Benjamin Bloom’s taxonomy of 
educational objectives (1956), and the items are arrayed from lower-order to higher-order thinking skills. 
Perhaps no other classification scheme has been as influential and continues to contribute to as many 
educational fields (e.g., Zheng et al., 2008 on the content of AP Biology, undergraduate instruction, the 
MCAT, and medical school). 

Factor 6: Campus Climate for Diversity 

Factor 6 appeared as part of the core factor structure in 2008 when eight items asking about respect for 
students and freedom of expression of beliefs and ideas were moved from a randomly assigned module 
to the universally assigned core item set. The subfactor structure is particularly remarkable in that it 
differentiates between the expression of beliefs, treatment of students for factors perceived to be matters 
of choice, and factors beyond choice (Table 6). For example, items about gender, economic or social 
class, race and ethnicity, and sexual orientation are differentiated from religions and political beliefs. The 
dual loading of sexual orientation on choice and beyond-choice dimensions was interesting, and likely 
reflected society’s multiple understandings about the nature of less common sexualities. 

Factor 7: Academic Disengagement (Inverted Scale) 1 

While the concept of Academic Disengagement is straightforward, computing and reporting of Academic 
Disengagement are more complicated (Table 7). Skipping class, being unprepared, not reading material, 
partying, and watching TV are all examples of activities that will interfere with academic involvement 
whether or not they are associated with lower academic performance. A student exhibiting high levels of 
these behaviors has less time for academic matters. An area of special interest for future research is the 
presence of activities generally considered to be healthy for students on several levels: exercise, 
involvement in student clubs and organizations, and recreational or creative interests and hobbies. 

Factor 8: Quantitative Professions 

From its full inception in 2006, the SERU/UCUES questionnaire has been atypical because it has items 
that better reflect the undergraduate experience of science, business, and engineering students (see 
Table 8). The subfactors are ratings of quantitative skills, both current and gained, and reasons for 
selecting a major that are reward-driven: socially, personally, and monetarily. 

Factor 9: Time Factor 

The items of Factor 9 have never been part of factor solutions. They are items that the first panel, in 
2006, judged to be especially important to universities and that needed to be formed according to 
university interest in student employment and academic time (see Table 9). 
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